Model engineering : balancing between virtuality and reality

Model engineering concerns the development of models of complex systems. This modeling is performed for a variety of reasons, such as system behavior prediction, system optimization or system construction. Model engineering requires a modeling framework that includes a language to represent the model and a set of techniques to analyze the model. Kees van Hee has made many actual models and developed modeling frameworks. In the lecture he will present an overview of this field, based on his experiences

Adaptive control of specially structured Markov chains

We consider Markov decision processes where the state at time n+1 is a function of the state at time n, the action at time n and the outcome of a random variable Y_{n+1}. The random variables Y_1, Y_2, Y_3, ... are independent and identically distributed with an incompletely known distribution. The class of problems considered includes the linear system with quadratic cost and a simple inventory control model. The minimal Bayesian expected total cost is determined or approximated. The strategy that takes, at each time, the action that is optimal if the estimated distribution is the true distribution, is studied

The policy iteration method for the optimal stopping of a Markov chain and applications to a free boundary problem for random walks

In this paper we study the problem of the optimal stopping of a Markov chain with a countable state space. In each state i the controller receives a reward r(i) if he stops the process and he must pay the cost c(i) otherwise. We show that under some conditions, the policy iteration method, introduced by Howard, gives the optimal stopping rule in a finite number of iterations. For random walks with a special reward and cost structure the policy iteration method gives the solution of a free boundary problem. Using this property we shall derive a simple algorithm for the determination of the optimal stopping time of such random walks

Markov decision processes and strongly excessive functions

AbstractStrongly excessive functions play an important role in the theory of Markov decision processes and Markov games. In this paper the following question is investigated: What are the properties of Markov decision processes which possess a strongly excessive function? A probabilistic characterization is presented in the form of a random drift through a partitioned state space. For strongly excessive functions which have a positive lower bound a characterization is given in terms of the lifetime distribution of the process.Finally we give a characterization in terms of the spectral radius